feat: add OCI 1.1 Referrers API support with configurable distribution#1691
feat: add OCI 1.1 Referrers API support with configurable distribution#1691anithapriyanatarajan wants to merge 1 commit into
Conversation
|
@anithapriyanatarajan: The label(s) DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
7ff8190 to
d757d46
Compare
|
@tektoncd/chains-collaborators @arewm - Please review and share your comments. Thank you |
d757d46 to
c429a7e
Compare
|
Detailed steps to test and observations are captured here for reference - https://gist.github.com/anithapriyanatarajan/b112638e7b3d78ee6638c6fec4f51bc8 Observed Gaps, which would be analyzed further with appropriate follow ups:
Registry readiness for OCI 1.1 is much critical to realize the benefit of this PR for users. |
c429a7e to
4d43d45
Compare
| Signature: []byte(signature), | ||
| Cert: []byte(storageOpts.Cert), | ||
| Chain: []byte(storageOpts.Chain), | ||
| PublicKey: storageOpts.PublicKey, |
There was a problem hiding this comment.
PublicKey is added here for the protobuf-bundle path, but Content on the line above is still nil.
uploadSignature correctly passes Content: rawPayload, but uploadAttestation leaves it nil. This is fine for storeLegacy and storeWithReferrersAPI (DSSE mode) which only use req.Bundle.Signature.
But storeWithProtobufBundle calls buildDSSEEnvelope(req.Bundle.Content, ...) and cbundle.MakeNewBundle(..., req.Bundle.Content, ...) both receive nil.
Verified on ghcr.io: the protobuf-bundle attestation's dsseEnvelope has no "payload" key, while the DSSE-mode attestation correctly has ["payload", "payloadType", "signatures"].
There was a problem hiding this comment.
Thank you for the detailed investigation. This is fixed now and verified with quay and ghcr. Please review and mark it as resolved if you are convinced.
4d43d45 to
ee93f82
Compare
| if err != nil { | ||
| return nil, errors.Wrap(err, "attaching attestation to entity") | ||
| } | ||
| if err := ociremote.WriteAttestationsReferrer(req.Artifact, newImage, ociremote.WithRemoteOptions(s.remoteOpts...)); err != nil { |
There was a problem hiding this comment.
Just want to confirm, here we are using req.Artifact, which drops the override of repo and ignores the storage.oci.repository. Should we use the overridden repo instead?
|
More details on why we are doing this change - sigstore/docs#411 & kubernetes-sigs/tejolote#639 |
|
@anithapriyanatarajan |
|
@anithapriyanatarajan can you add e2e tests ? |
ee93f82 to
904528b
Compare
@jkhelil - Yes, storage.oci.format=referrers-api can be safely configured without knowing the registry's OCI 1.1 support status. The fallback to the Referrers Tag Schema is handled automatically by go-containerregistry (the underlying library used by cosign and vendored in Chains). If the registry natively supports the Referrers API (e.g., Quay), it is used directly. If not, go-containerregistry automatically falls back to storing referrers as an OCI Image Index tag, per the OCI distribution spec mandate (spec lines 777–806). Since this fallback is transparent at the library level, Chains does not need to implement its own fallback logic, and users do not need to manually revert to the legacy storage format based on registry capability. I have verified this behavior with quay and ghcr. For quay it takes the referrers API path while for ghcr it takes the referrers tag path. Will share evidences after fixing a few other findings. Thank you |
904528b to
78d47da
Compare
78d47da to
276c83e
Compare
| Referrers schema. If the registry supports the Referrers API natively, cosign | ||
| uses it. If it doesn't, cosign falls back to the spec's **referrers tag schema**: |
There was a problem hiding this comment.
This isn't necessarily the case. I don't think cosign would have used the referrer's API in v2 unless an additional option was passed.
But maybe you have tested things more than me, so your content might be more authoritative.
| - In **legacy** mode, the attestation is a **DSSE** envelope. This is what cosign | ||
| has always written and what existing tooling verifies. | ||
| - In **referrers** mode, Chains writes the attestation as a **Sigstore protobuf | ||
| bundle**. |
There was a problem hiding this comment.
Maybe you want to differentiate by what is the default configuration for cosign versions?
|
|
||
| These are interoperability notes, not bugs in Chains. | ||
|
|
||
| 1. **`storage.oci.repository` is ignored.** A referrer has to live next to the |
| 5. **Concurrent writes can race on fallback registries.** Without the native API, | ||
| the index tag is updated with read-append-write, so simultaneous writes to the | ||
| same image can drop an entry. A registry with the native Referrers API avoids | ||
| this. |
There was a problem hiding this comment.
This refers just to the legacy mode?
276c83e to
93d718e
Compare
41446d2 to
0dd4ea1
Compare
Signed-off-by: Anitha Natarajan <anataraj@redhat.com> Co-authored-by: Copilot <claude-sonnet@users.noreply.github.com>
0dd4ea1 to
049070b
Compare
Done |
| | `storage.gcs.bucket` | The GCS bucket for storage | | | | ||
| | `storage.oci.repository` | The OCI repo to store OCI signatures and attestation in | If left undefined _and_ one of `artifacts.{oci,taskrun}.storage` includes `oci` storage, attestations will be stored alongside the stored OCI artifact itself. ([example on GCP](../images/attestations-in-artifact-registry.png)) Defining this value results in the OCI bundle stored in the designated location _instead of_ alongside the image. See [cosign documentation](https://github.com/sigstore/cosign#specifying-registry) for additional information. | | | ||
| | `storage.oci.repository.insecure` | Whether to use insecure connection when connecting to the OCI repository | `true`, `false` | `false` | | ||
| | `storage.oci.distribution-method` | Controls how OCI signatures and attestations are attached to images in the registry, and implicitly the payload encoding: `legacy` uses tag-based storage with DSSE payloads, `referrers-api` uses the OCI 1.1 Referrers API with Sigstore protobuf-bundle attestations. See [OCI Artifact Distribution (Referrers)](oci-artifact-distribution-format-referrers-schema.md) for details. | `legacy`, `referrers-api` | `legacy` | |
There was a problem hiding this comment.
I suspect that users will care more about the format whether that is legacy or sigstore bundle formats. It is just that sigstore bundles also happen to require the referrer's api. I don't know how much reframing of the document that would require.
There was a problem hiding this comment.
referrers-api is specific to the OCI storage backend. All other backends (GCS, Firestore, Grafeas, docdb, Archivista, Tekton annotations) continue to use DSSE as-is. This PR does not change them.
As long as cosign verify-attestation and cosign verify-blob-attestation continue to support DSSE-encoded attestations, the current split is clean. OCI users can opt into the new bundle format via referrers-api, while everything else stays on DSSE. I've raised a clarification with the cosign/sigstore community on this point: https://sigstore.slack.com/archives/C01PZKDL4DP/p1782872895452699.
However I would like to get views from @tektoncd/chains-collaborators on a broader question: should Chains eventually move all storage backends to the Sigstore protobuf bundle format (which mandates protobuf serialization), or is the current per-backend split the right long-term architecture?
There was a problem hiding this comment.
@arewm - just to reconfirm, from a user's perspective, choosing referrers-api defaults to sigstore bundle. There is no referrer api support DSSE format. So with that clarification, I hope this PR is complete. Rest of the Backends, I prefer to handle in separate PR. Do you have any further concerns with this? Thank you.
There was a problem hiding this comment.
Yes, I agree with that statement but if we expect that users' primary concern for supportability is the format (i.e. bundle vs DSSE) then we should provide that as the toggle. Many registries support the referrer's api v1.1. Again, this is just my assumption. I care about the referrer's API but others might not. It seems reasonable to make the toggle most apparent for the more-likely-to-break change.
There was a problem hiding this comment.
😸 Shall we rename storage.oci.distribution-method to something more format-centric, like storage.oci.bundle-format with values legacy (DSSE + tag-based) and sigstore-bundle (protobuf bundle + referrers API). The referrers API becomes an implementation consequence of choosing the bundle format, rather than the primary toggle. This also aligns better with how users will reason about cosign v4 compatibility.
Would that naming satisfy your concern? Happy to do the rename in this PR if it gets us to approval.
we could extend a similar configuration for each non-oci backend as we progress with the changes. storage.gcs.bundle-format, etc.,
There was a problem hiding this comment.
If you were to name as bundle-format, I assume that would just be a true/false configuration? If you want the configuration to be clearly legacy vs bundle, a storage-format or encoding-format might make more sense? Instead of having legacy/bundle options, it might also be clearer to have it be DSSE/bundle as that is the actual format?
So maybe the suggestion would be storage.oci.format with available options of DSSE (default for now) and bundle?
|
/approve |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: jkhelil The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@anithapriyanatarajan: PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
|
@jkhelil - I would like to reiterate to see the feasibility of accommodating #1691 (comment). Hence marking this as work in progress. |
NOTE TO REVIEWERS: Originally this PR intended to allow a user-configurable approach for both distribution and serialization. During later review and iterative testing it became clear this adds needless complexity for users, and we additionally discovered via sigstore/cosign#4841 that the
referrers-api+ DSSE combination is not supported in cosign. The PR therefore distills to two simple scenarios behind a single config knob, summarized below.Changes
Chains currently stores all OCI signatures and attestations using a tag-based scheme (
sha256-<digest>.sig/.att). This creates extra tags in the registry for every signed artifact. This PR adds support for the OCI 1.1 Referrers API as an alternative storage mechanism.New configuration key
A single new key controls where artifacts are stored. The serialization encoding is tied to it automatically (not separately configurable):
storage.oci.distribution-methodlegacy,referrers-apilegacyPossible Scenarios:
distribution-methodlegacyreferrers-api.sig/.atttagslegacy+ protobuf-bundle andreferrers-api+ DSSE are intentionally not offered: cosign's verification for referrer-stored attestations expects the protobuf bundle, andreferrers-apiexists precisely to move away from the legacy tag layout, which has downsides:This keeps the surface to one knob and avoids a confusing matrix of combinations, some of which no tool can verify.
Backward compatibility
All existing deployments with no OCI distribution config set continue to behave identically (
legacy+ DSSE).Implementation
config.go—OCIStorageConfiggains aDistributionMethodfield; new constants (OCIDistributionLegacy,OCIDistributionReferrersAPI) and avalidateOCIDistributionMethodvalidator. Defaults tolegacywhen unset.options.go— NewWithDistributionMethod()option function (replaces the formerWithFormat()); houses thereferrersRepoOverrideIgnoredhelper.attestation.go/simple.go— Dispatch keys offdistributionMethod;referrers-apiwrites attestations as a protobuf bundle viaociremote.WriteAttestationNewBundleFormat, while signatures use cosign's native signature referrer.legacy.go— Backend passesWithDistributionMethodwhen constructing storers.signing.go/signing/iface.go—signing.BundleandStorageOptsgain aPublicKey crypto.PublicKeyfield;PublicKey()extraction after signing is non-fatal (logs a warning, storage continues) so KMS signers don't break the legacy path.Tests
config_test.go— Tests the default, valid/invalid values fordistribution-method, and the validator.oci_test.go— TestsWithDistributionMethod, empty defaults, the Backend config, and thatstorage.oci.repositoryoverride is ignored in referrers mode.store_test.go— UpdateddefaultStorageto reflect the struct.Docs
oci-artifact-distribution-format-referrers-schema.md(new) — Tutorial-style page explaining the Referrers schema, how cosign and Chains enable it, why referrers + protobuf (not DSSE), registry compatibility, and verification examples.config.md— Newstorage.oci.distribution-methodrow in the storage configuration table.Submitter Checklist
As the author of this PR, please check off the items in this checklist:
functionality, content, code)
Release Notes
/kind feature
Co-Authored-by: Copilot ( Claude Sonnet 4.6)